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Abstract 

The purpose of the Technology and High School Success (THSS) initiative was to encourage 
innovative strategies focused on improving provincial high school completion rates, using 
technology and student-centered learning to engage student interest. The primary purpose of this 
paper is to report on barriers that impede systemic, effective and sustainable technology 
integration within schools. Even with teacher and administrative support and commitment for 
THSS, evaluative research indicated minimal change in system capacity as a result of the 
initiative. Three primary barriers to program sustainability were: 1) schools and school districts 
did not leverage the opportunity to revisit their existing vision(s), 2) schools and school districts 
did not use data to make changes, and 3) limited access to technology. 

Resume 

Le but de Tinitiative « Technology and High School Success (THSS) » etait d’encourager des 
strategies novatrices visant a ameliorer le taux d’achevement des etudes secondaires dans la 
province de TAlberta en utilisant la technologie et un apprentissage centre sur l’eleve a (in de 
susciter (interet des eleves. L’objectif principal de cet article est de rendre compte des obstacles 
entravant (integration systemique, efficace et durable de la technologie dans les ecoles. La 
recherche evaluative a revele des changements minimes dans la capacite systemique a la suite de 
(initiative, et ce, malgre Tappui et Tengagement des enseignants et des administrateurs envers le 
THSS. Les trois obstacles principaux a la viabilite du programme sont les suivants: 1) les ecoles 
et les districts scolaires n’ont pas profite de Toccasion pour revoir leur(s) vision(s), 2) les ecoles 
et les districts scolaires n'ont pas utilise les donnees pour effectuer des changements, et 3) (acces 
limite a la technologie. 
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Introduction 

Research has shown that people learn best when trying to do things that are challenging and of 
deep interest to them, reflecting the close interplay of affect and cognition in the development of 
capacity (Csikszentmihalyi 1990, 1997; Jacobsen, Friesen & Saar, 2010; Kuh, 2001, 2003; 
Willms, Friesen & Milton, 2009). A recent Canadian study (Willms, Friesen & Milton, 2009) 
provides compelling evidence that schools differ considerably in demonstrated levels of student 
engagement, and that differences among schools have less to do with students’ backgrounds than 
they do with school policies and practices, and in particular, the learning climate established by 
teachers in the classroom. For example, Means, Toyama, Murphy, Bakia, and Jones (2009), in a 
meta-analysis of over 175 studies, claimed that blended learning environments produced stronger 
learning outcomes than did classes with in-person teacher instruction. In many of these 
situations, student-directed, interactive, and collaborative learning experiences were more 
effective than teacher-directed, whole class instructional methods alone (Lee, Linn, Varma & 

Liu, 2010; Jacobsen, 2010; Means, et al., 2009). It might be tempting to suggest from findings 
like these that it is the introduction of technology itself into the classroom that is responsible for 
the stronger learning outcomes; however, the reality is likely more nuanced. 

Major technology initiatives are not uncommon in K-12 environments, where funds are provided 
for buying various types of instructional technology in the hopes of providing transformational 
experiences to students. In many of these cases, the approach often seems to consist of trying to 
place as much technology as possible in schools in the hope that teachers and students will derive 
some benefits from the technology simply by its addition (OECD, 2010). Yet, despite the 
number of these projects and the expected benefits of using technology in the classroom, the 
hoped for transformations do not always take place (Cuban, 2006). When transfonnation does 
take place it is usually limited, incremental, and variable, perhaps having more to do with general 
teaching practice than technology use (Weston & Bain, 2010). Given the time and resources 
required for systemic change, the lack of sustainability can prevent projects ever reaching scale. 
So every time a new technology related initiative begins the process often starts over from the 
beginning (Law, 2009). 

In 2007 the provincial government of Alberta, Canada issued a Call for Proposals to all publicly 
funded school districts and charter schools for the purchase of technology and to demonstrate 
how the innovative use of technology could improve the student learning experience at the 
secondary level (mostly grades 9 through 12). In total, 24 school districts and/or charter schools 
were successful in receiving funding. As part of this initiative each school district was required 
to conduct its own research project. In addition, the authors were contracted to conduct a macro¬ 
level, provincial evaluation of the initiative during the final two years of the project (2008-2010) 
and to provide information back to the government and the public regarding the general results 
of the program. A key component in understanding how technology can be used to increase 
student success is to gather infonnation regarding barriers to systemic, effective and sustainable 
technology integration. In this paper we present selected findings and implications from this 
macro-level evaluation of the provincial initiative examining the link between technology and 
high school success. 
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Method 

Data Collection 

We used a mixed method case study approach for this study. A mixed methods approach was 
employed for two reasons: triangulation which increased convergent validity through the use of 
multiple measures of similar underlying concepts; and complimentarity which means that 
different aspects of what was being examined could be evaluated using different methods 
(Green, Caracelli, & Graham, 1989). Because the phenomena of interest in this investigation 
were complex (23 school districts involving over 22,000 students and 420 teachers at over 70 
schools) we needed a range of research methods to capture and describe the complexity of each 
case, and to facilitate cross-case synthesis and explanation building (Yin, 2006). 

We collected data from three sources: (1) online surveys, including a student engagement survey, 
a student technology use survey, and a teacher technology use and educational practice survey; 
(2) interviews and focus groups with students, teachers, team leads, technical support staff, 
school administrators, and district administration; and (3) field notes and classroom observations 
from site visits to the school districts. 

Data Collection and Analysis 

Data were collected throughout the 2 years of the macro-level evaluation of the program, from 
September 2008 to June 2010. All qualitative data were processed through an iterative process of 
reading, rereading, coding and review. The two student surveys, assessing students’ perceptions 
of intellectual engagement (based on “What Did You Do in School Today”, Canadian Education 
Association) and students’ perceptions of technology were analyzed quantitatively (n=2,433). 
Teachers completed surveys regarding their instructional planning and classroom practises 
(n=294). Eighty-two interviews and focus groups with students, teachers, support staff and 
administrators were coded and subjected to content analysis to discern patterns of experience and 
develop themes. Forty classroom observations focused on discipline, instructional practises, 
student engagement and student interactions. These observations were aggregated and analyzed 
using a combination of descriptive statistics and qualitative content analysis. 

Participants 

Of the 2,433 students who completed the survey, 47.7% were female and 50.5% were male 
(1.8% chose not to respond). All students were in grades 9 through 12 with the majority being in 
grades 10-12. Student focus groups were conducted with students in grades 9 through 12. 

When teachers were asked about years of teaching experience, the most frequently selected 
category was more than 20 years (21.8 % of responses); however, responses were spread fairly 
evenly across categories of teaching years (less than 3 years - 13.3%, 3 to 5 years - 14.6%, 6 to 
9 years - 18.0%, 10 to 14 years - 16.3%, and 15 to 19 years - 12.9%). Forty-three percent of 
teachers indicated that they have been using instructional technology in their classrooms for 2 to 
5 years. 
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Results and Discussion 

Data from the participating school districts were analyzed to detennine the progress that the 
school districts, as systems, were making progress towards creating innovative learning 
environments (Friesen & Lock, 2010). The components for the system were derived from the 
research literature (Bransford, Brown & Cocking, 2000; Friesen & Lock, 2010; Hargreaves & 
Shirley, 2009; Jenkins, Purushotma, Weigel, Clinton & Robison, 2009; Lemke, Coughlin, 

Garcia, Reifsneider & Baas, 2009; OECD, 2006, 2008, 2009a, 2009b; Sawyer, 2006, 2008; 
Scardamalia & Bereiter, 2006; Scardamalia, Bransford, Kozma & Qyekknakzm, 2010). In 
examining the data, we identified three barriers that impacted school districts’ and schools’ 
abilities to realize systemic, effective, and sustainable technology use: (1) school districts and 
schools did not leverage the opportunity to re-examine their vision, (2) there was little evidence 
of school districts and schools using research evidence to inform or effect changes to their 
practice, and (3) access to effective technology was still a limiting factor. The focus, generally, 
seemed to be on acquiring technology and increasing the technological infrastructure rather than 
on investing in the human infrastructure and examining how that technology could be used most 
effectively to impact changed teaching practices and to create greater learning opportunities for 
students. Technology was often seen as the solution rather than a means to an end. 

Re-examining Vision for Learning With Technology 

When examined in aggregate, the survey and interview data indicate that many school districts 
are at a beginning level in formulating a contemporary shared vision that incorporates the new 
realities in teaching, learning, and leading. During interviews, school leaders and teachers often 
referenced jurisdiction planning documents; very few were able to describe how the jurisdiction 
goals for learning with technology were created, and no participant indicated that they had had a 
voice in developing the jurisdiction’s vision for learning with technology. Interviews with 
jurisdictional leaders indicated that school districts are also at beginning levels in aligning the 
system to the new vision statements about learning with technology. 

As part of this initiative, we visited schools within participating districts twice over the course of 
the research project. During each visit we observed classrooms to see how the technology was 
being used. The schools chose which classrooms and teachers we visited. We assume that they 
selected teachers whose practices were consistent with the school’s perspective regarding 
technology use and who were considered successful at using technology in their teaching and for 
student learning. Based on our observations, it appears that the schools’ perception of what the 
research was trying to document was visible and heavy technology use by teachers - which is 
what we saw and what students reported. We were invited into classrooms where teachers were 
using technology, but saw little evidence of technology being used in support of increased 
student engagement through teachers’ use of innovative instructional strategies or practices. 

One barrier to meaningful and authentic use of technology for knowledge building appears to be 
the assumption that all that needs be done is make technology available to teachers, to put it in 
classrooms and teachers will figure out the rest (Cuban, 2006; OECD, 2010; Weston & Bain, 
2010). Another aspect of this barrier appears to be the enduring image of what successful 
technology use by teachers is - which is linked to an infonnation delivery approach, versus a 
student centered, assessment for learning approach. So, what we observed was the values-in- 
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action; teachers selecting and controlling the technology and the content. What is not clear from 
our observations, though, is whether this is a limitation of the teachers understanding of best 
practices with technology for learning or the schools and the culture within which the teachers 
work. 

In analyzing the different ways leaders, both district and school, had taken up both challenge and 
opportunity that this initiative presented them, it was noted that most district and school leaders 
adopted a fairly linear approach to a complex problem. The linear approach was marked by 
articulating or pointing to a vision and attempting, or not, to align this new initiative to a pre¬ 
existing vision. There was little evidence of systemic efforts to revisit or rearticulate the school 
district or school vision based on this new Technology and High School Success initiative. When 
we spoke to leaders and teachers very few could articulate the vision for the initiative and fewer 
still were able to connect actions they were taking with the goals and expected outcomes of the 
project. This approach appears not to directly address sustainability as it seems to assume that 
existing methods and approaches are adequate and, therefore, sustainability is not directly 
addressed. 

It appeared that many school districts and schools did not leverage the opportunity that this 
project presented or saw the need to (at least initially) revisit their existing technology vision. As 
one teacher noted, expressing frustration regarding the lack of an updated vision, “.. .the fact that 
things like email sites and blogs and wikis are blocked by our jurisdiction from the beginning 
speaks to the vision that they seem to have, right?” Instead, many district and school level 
administrators reported that they viewed this project as an add-on. A few reported that they felt 
they were suffering from initiative overload in their school districts or schools and a project such 
as this one just added one more thing into an already overburdened list of initiatives. It should be 
noted, however, that all of the school districts in this project had volunteered to submit a 
proposal for this initiative; participation in THSS was not mandatory. 

In some schools it appeared that there was a disjoint between who championed the proposal and 
the people who were charged with carrying it out. In a number of schools the person or people 
who championed the project were no longer directly involved. In others, there were issues and 
problems that took priority over the technology (but still impacted the effective use of 
technology). For example, one project leader felt that the project would possibly have not 
succeeded if he had not been there to champion it, “And if it wasn’t for me, and I’m not tooting 
my own horn, but if I didn’t push it, push it, push it, it probably wouldn’t—I’m glad I didn’t 
leave. It may have fell flat.” The schools and principals were supportive, but there was often a 
gap between the verbal support and how this actually translated into meaningful professional 
development for teachers, and a robust and reliable technology infrastructure. This gap could be 
a systemic problem as changes in roles and responsibilities can be expected over the course of 
any project such as this one. 

There have been many promises regarding the potential of technology to revolutionize education, 
to reach out to disaffected students and to provide a more level playing field for all children 
(Edwards, 2003). The reality in schools, however, has not always matched the hype. Though our 
research does seem to indicate that students felt that they had adequate access to technology in 
the classroom (see Figure 1), it was the teachers who were primarily using the technology; in 
most cases in teacher-led discussions (see Figure 2). Rather than the use of technology leading to 
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more student-centered or discovery learning to more directly engage student learning, it appears 
that the technology was being used to do the same things that teachers were already doing. 


Interactive whiteboards (e.g., SMART 
boards) n=2372 

Computer labs (during class) n=2393 

Computer labs (outside of class time) 
n=2374 

Mobile computers (e.g., laptops) n=2376 
Computers in the classroom n=2374 
Video conferencing equipment n=2373 



66.9 

57.1 



■ 48.! 

51.9 




I 60.4 

39.6 




1 

64.8 

35.2 



81.3 

18.7 




-100%-75% -50% -25% 0% 25% 50% 75% 100% 


■ Disagree Agree 


Figure 1. Students’ opinions regarding the adequacy of certain technologies 
in the classroom. Students agreed that they had adequate interactive whiteboards, 
but were less certain or disagreed regarding several other technologies such as mobile 

computers or computers in the classroom. 


Creating schools and school districts for today requires educators who are attuned to the 
demands of a knowledge society (OECD, 2010; UNESCO, 2005). Acknowledging this fact will 
mean that schools will need to broaden their focus from managing infonnation exchanges to 
engaging learners, all learners—youth and adult alike—in collaborative knowledge building 
activities (Bransford, et ah, 2000; Gilbert, 2005; Hargreaves, 2003; Hargreaves & Shirley, 

2009; Jardine, Friesen & Clifford, 2006; Papert, 2004; Sawyer, 2008; Scardamalia & Bereiter, 
2003; UNESCO, 2005; Wagner, 2006). From within school structures and processes designed to 
meet the needs of the industrial past, educational leaders are called upon to invent and design 
new learning environments and new education systems to address our contemporary society. As 
evidenced in this study, it is becoming clearer to many school districts and school leaders that 
simply adding technology to the current onc-sizc-fits-all system will not get their school districts 
to where they need to go. Based on the one-size-fits-all perspective, it should not be surprising 
that some teachers and leaders see the technology as an add-on to their existing workload, rather 
than an opportunity to rethink practices and learning designs. 
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Watching/Listening to the teacher present 
material to the entire class using technology 
n=2366 


24 


76 


Working alone n=2378 


30 


70 


Working in pairs n=2363 


45 


55 


Working in small groups n=2361 


57 


43 


100% 50% 0% 50% 


100 % 


■ Infrequent use Frequent use 


Figure 2. Students were asked to indicate how and with what frequency technology 
was used in the classroom. Most students indicated that when technology was used 
in the classroom, it was most often used by teachers to present material to the 
whole class. Almost as frequently, students indicated that they used technology 
in the classroom when working alone. 


Evidence-based Practice 

In this study, we found similar evidence to what has been found in other studies (e.g., Drill, 
Miller, & Behrstock-Sherratt, 2013); we found that while teachers and principals recognized the 
theoretical benefit of using research in their work, in practice, they were unlikely to use it. While 
it appeared from their comments that many of the leaders in the current study saw the need to 
collect evidence as the initiative unfolded (which was a requirement of the Provincial 
government funding), very few actually used the data they collected in any discernible way to 
inform the next steps of the initiative. As one district administrator indicated when asked 
whether there had been any attempt to evaluate the impact of the initiative, “We haven’t done 
any real type of tests or anything to see—pretests, post-tests—to see have kids improved, not 
improved, stayed the same. We haven’t done anything like that.” 

In a recent commentary, Baker and Weiner (2012) advocate for the use of high quality research 
to increase productivity and efficiency in the educational system. Baker and Weiner propose a 
national consortium to bring together high quality research for US schools. Despite advocates 
such as Baker and Weiner, research data is still seldom used in education. Davies (1999) 
suggests two reasons why research has yet to find a foothold within education. First, 

“educational activity is often inadequately evaluated” (p. 109). Second, “research and evaluation 
studies that do exist are seldom searched for systematically, retrieved and read, critically 
appraised for quality, validity and relevance, and organized and graded for power of evidence” 
(p. 109). Based on interviews with jurisdiction and school leaders, and classroom teachers, the 
use of research to inform innovative uses of technology for learning, and the collection and use 
of data on the success of the initiative is ad hoc at best. 
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There are many potential benefits to effectively using research and data in education. For 
example, creating 21st century systems requires systems thinking. Twenty-first century systems 
thinking requires the system 1) to pay attention to what is emerging and evolving which 
necessitates collecting evidence along the way and making decisions informed by both research 
and evidence, and 2) to create structures and processes that are adaptable. These leaders 
understand at the deepest level that a knowledge-building organization is created through its 
connections and relationships, not its flowchart. Research can facilitate this process leading, 
potentially, to better, more informed decisions. A focus on research can also lead to a mindset of 
inquiry, not certitude. Using this approach, leaders could set in motion short-tenn processes 
towards the vision, collecting relevant and timely evidence at every step throughout the project, 
which they then use to monitor progress and create the next steps towards the vision, fully 
responsive to what was emerging. 

Competencies and capacity to enact the changes required to make progress towards building the 
capacity for a 21 st century system requires leadership practices for “knowledge driven 
organization^] which demand innovation and creativity from all employees" (Murgatroyd, 2010, 
p. 4). An additional question that should be considered but was not part of this study is whether 
school districts and schools have the capacity to understand, interpret and use research findings 
even if research is being done; as was the case in this instance. 

Recent efforts to use learning analytics may be an opportunity to begin to systematically 
incorporate data into teaching and learning. While this seems to be a move towards more 
evidence-based decision-making, there are still concerns regarding how this data is being used 
(Buckingham Shum & Ferguson, 2012). Simply having data is no guarantee that it will be used 
effectively. According to Booth (2012) 

... even though learning analytics offers powerful tools and practices to improve the work 
of learning and assessment, well-considered principles and propositions for learning 
assessment should inform its careful adoption and use. Otherwise, learning analytics risks 
becoming a reductionist approach for measuring a bunch of “stuff’ that ultimately 
doesn’t matter. (Learning Analytics: The New Black, para. 2) 


Access 

Within this context, access to technology was found to be a limiting factor in effective use. 
Firewalls, content filters and Internet throttling constrain reliable teacher and student access to 
high quality online resources (see Figure 3 and Table 1). The selection of and access to high 
quality resources for student learning can directly affect pedagogy, and decisions about 
resources, physical, analogue or digital, should be strongly influenced by teachers and students. 
This study revealed a frustrating disconnect between increased access to technology and 
networks in secondary classrooms and the actual ability of classroom teachers and students to 
benefit from and take advantage of this access. From classroom observations, survey and 
interviews, a clear finding with regard to blocked or throttled access to online resources 
emerged: Content and internet site filters, network firewalls and choked Internet speeds prevent 
students and teachers from accessing high quality online resources during class time. Further, 
filters and firewalls disrupt and often disable attempts to foster online collaboration. This 
emphasizes the disconnect that can occur between teachers’ expectation and desire for open 
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access to online resources and administration’s view on what is required for teaching and 
learning in 21st century classrooms. 



Figure 3. Teachers (n=294) were asked to indicate whether they have 
had access difficulty due to firewalls or blocking. The majority of teachers 
indicated that they had experienced access difficulty. 


Table 1: Teachers’ Comments Regarding the Types of Access Difficulties 


Comment category 

Number of 
comments 

Exemplary comment(s) 

Blocked websites 

89 

Almost all useful sites are blocked, especially for the 
kids. It should be the teachers responsibility to 
monitor sites, they should not be firewalled (except 
for major no no's like pornography). 

YouTube related 

28 

Some videos on You-tube which are curriculum 
relevant have been blocked due to restrictions even 
though I have reviewed them at home and they do 
not breach any access guidelines. 

Technical difficulties 

26 

Can't open my school email or open the server. 

Access 

19 

Because they set firewalls at the school are set by 
tech people outside of the teaching profession it 
seems as though the filters are really weird. 

Sometimes I can’t access simple images or websites 
that I might find useful for teaching or ones that I 
found at home and yet can't access here. 

Student content sites 

3 

Unable to access student content. 

Other 

17 

The filter at times is very cumbersome. 
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Conclusion 

This initiative provided to us a wealth of data to infonn future initiatives that investigate the li nk s 
between technology, student engagement and school success. For example, what lessons can be 
learned to increase the transferability and sustainability of technology-supported innovations in 
schools in the future? What do school districts and schools say they need to sustain and transfer 
policy innovations? 

Unless the processes around technology initiatives change, it is unlikely that the desired 
sustainability will occur. In this study, consideration of sustainability often occurred towards the 
end of the initiative once the money had all been spent. Leaving consideration of sustainability to 
the end of an initiative will be a significant challenge since all the stakeholders must take 
responsibility and ownership in this process. There also needs to be a realistic understanding of 
what the technology can and cannot offer (Selwyn, 2010). When technological innovations fail 
to create systemic change, the easiest solution often seems to be to lay the blame on the 
technology, to claim that the technology used was inadequate to the task, hoping new 
technologies might solve the problem thereby relieving others from taking responsibility. 
However, as Weston and Bain (2010) claim, “.. .like so many problems in changing venerable 
institutions, it too often is easier simply to protect the status quo and blame the innovations or the 
innovator” (p. 9). It is not the technology, per-se, that is or is not having an impact on student 
engagement, but rather the teaching and learning practices that are enacted and supported to 
leverage the benefits of technology in new learning designs. While we believe that having 
technology in the classroom can provide many opportunities for enriched learning, having 
technology in the classroom is not, by itself, going to cause students to be more engaged in their 
own learning any more than having a garden is likely to make one a gourmet chef. What we had 
hoped to document in this initiative was convincing evidence that greater access to classroom 
technologies enabled teachers to make meaningful changes to how they designed and enacted 
learning opportunities in their classrooms. While we documented pockets of such innovation in 
several classrooms, the impact of the Technology and High School Success initiative was not 
widespread, did not result in systemic changes in vision or strategy, and has not resulted in 
sustainable changes to learning opportunities for Alberta students. 

As researchers who have worked on a number of these initiatives, it is striking to us the 
similarity of these projects in terms of goals and outcomes, despite the attempts by the provincial 
ministry to try to improve the chances of success. Through this study we have been able to 
examine 3 barriers that make it difficult to create systemic change: 1) schools and school districts 
did not leverage the opportunity to revisit their existing vision(s), 2) schools and school districts 
did not use collected data to make changes, and 3) limited effective access to technology. 

Though perhaps not exhaustive, addressing these barriers is a necessary step in ensuring that 
technological initiatives are systemic, effective, and sustainable. 
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